On-board light source calibration

ABSTRACT

An example method includes recording dark images on an image sensor on-board an orbital vehicle during flight, which include a first image recorded before the orbital vehicle is over a predefined location on the Earth and a second image recorded after the orbital vehicle is over the predefined location; and recording third and fourth images on the image sensor during flight based on illumination from a light source that is on-board, with the third image being recorded before the orbital vehicle is over the predefined location and the fourth image being recorded after the orbital vehicle is over the predefined location. A fifth image is recorded on the image sensor during flight while the predefined location on the Earth is visible to the image sensor. The fifth image is based on light from a ground-based calibration system. The light source is calibrated during flight based on the five images.

TECHNICAL FIELD

This specification describes examples of systems for calibrating, duringflight, light sources that are on-board an orbital vehicle, such as asatellite.

BACKGROUND

A satellite is used to record images of the Earth. The satelliteincludes image sensors (“sensors”) to capture images based on lightreflected from the Earth. The sensors are calibrated prior to launch ofthe satellite. However, over time, the performance of the sensors maychange. On-board calibration may be used to calibrate the sensors duringorbit. For example, lamps may be included on the satellite and used toperform relative calibration of the sensors. Relative calibrationincludes detecting normalized changes in the radiometric performance ofthe sensors over time and attempting to compensate for those changes.The lamps are calibrated prior to launch of the satellite; however, thelamps are also subject to changes in operation following launch andduring flight. Moreover, the lamps have not been used to performabsolute calibration of the sensors. Absolute calibration includesilluminating a sensor and converting an output digital number per sensorpixel into a representation of a physical phenomenon, such as radiance.An example of an absolute calibration system that is ground-based andnot on-board, and that may be used to calibrate on-board satellitesensors during flight, is the specular array for radiometric calibration(SPARC) system described in U.S. Pat. No. 8,158,929 (Schiller).Calibration techniques that use radiometric reference standards noton-board are referred to as vicarious calibration methods or systems.Vicarious systems such as SPARC create the potential for establishing aradiometric traceability path from the ground reference to an on-boardlight source through the Earth imaging system.

SUMMARY

An example method is directed to calibrating a light source that ison-board an orbital vehicle during flight. The method includes thefollowing operations: recording dark images on an image sensor on-boardthe orbital vehicle during flight, with the dark images including afirst image recorded before the orbital vehicle is over a predefinedlocation on the Earth and a second image recorded after the orbitalvehicle is over the predefined location on the Earth; and recording athird image and a fourth image on the image sensor during flight basedon illumination from the light source that is on-board the orbitalvehicle, with the third image being recorded before the orbital vehicleis over the predefined location on the Earth and the fourth image beingrecorded after the orbital vehicle is over the predefined location onthe Earth. A fifth image is recorded on the image sensor during flightwhile the predefined location on the Earth is visible to the imagesensor. The fifth image is based on light from a ground-basedcalibration system. The light source is calibrated during flight basedon the first image, the second image, the third image, the fourth image,and the fifth image. The method may include one or more of the followingfeatures, either alone or in combination.

The method may include generating a radiance map for a state of thelight source based on the first image, the second image, the thirdimage, the fourth image, and the fifth image. The method may includecontrolling operation of the light source to calibrate the image sensor.The orbital vehicle may include a shutter to open to the environment,and the light source and the image sensor may be behind the shutter.

Recording the dark images, the third and fourth images, and the fifthimage may include the following operations, which may or may not beperformed in the following order: closing the shutter to restrictenvironmental light from reaching the image sensor; recording the firstimage on the image sensor when the shutter is closed and the lightsource is off; turning the light source on to illuminate the imagesensor when the shutter is closed; recording the third on the imagesensors when the shutter is closed and the light source is illuminated;turning the light source off; opening the shutter to allow light from aground-based calibration system to illuminate the image sensor;recording the fifth image on the image sensor based on the light fromthe ground-based calibration system; closing the shutter to restrictenvironmental light from reaching the image sensor; turning the lightsource on to illuminate the image sensor when the shutter is closed;recording the fourth image on the image sensor when the shutter isclosed and the light source is illuminated; turning the light sourceoff; and recording the second image on the image sensors when theshutter is closed and the light source is off.

The light source may be among multiple light sources such as LED-basedlamps included on the orbital vehicle. Different combinations of themultiple light sources may define different states. The method mayinclude performing calibration for different states that include thelight source based on the first image, the second image, the thirdimage, the fourth image, and the fifth image.

Operations to perform the calibration may include the following:obtaining a dark bias of the light source based on the first and secondimages; removing the dark bias from the third, fourth, and fifth images;performing a calibration analysis of the image sensor based on the fifthimage having the dark bias removed to obtain gain coefficients forpixels in the sensor, where each gain coefficient is for converting adigital number for a pixel into a radiance value; determining averagedigital numbers for pixels in the image sensor based on the third andfourth images having the dark bias removed; and applying the gaincoefficients to the average digital numbers. Applying the gaincoefficients may include multiplying the gain coefficients by theaverage digital numbers.

The dark bias may include an average dark bias for multiple images. Themethod may include repeating the following operations for two or moredifferent spectral bands of the light: recording the dark images on theimage sensor on-board the orbital vehicle during flight, with the darkimages including the first and second images; recording the third andfourth images during flight; recording the fifth image on the imagesensor during flight while the orbital vehicle is over the predefinedlocation on the Earth; and generating the radiance map.

The ground-based calibration system may include a plurality of sphericalmirrors disposed upon a uniform background as at least one array ofreflective points, with at least two points of the array reflecting anintensity of directly incident sunlight.

An example system on an orbital vehicle may include an image sensor forcapturing images based on incident light; a light source to illuminatethe image sensor; a shutter that is controllable to allow environmentallight to reach the image sensor to prevent the environmental light fromreaching the image sensor; and a control system. The control system maybe configured—for example, programmed, constructed, and/or arranged—toperform operations that include the following: controlling the shutterto record dark images on the image sensor on-board the orbital vehicleduring flight, with the dark images including a first image recordedbefore the orbital vehicle is over a predefined location on the Earthand a second image recorded after the orbital vehicle is over thepredefined location on the Earth; and controlling the shutter to recorda third image and a fourth image on the image sensor during flight basedon illumination from the light source that is on-board the orbitalvehicle, with the third image being recorded before the orbital vehicleis over the predefined location on the Earth and the fourth image beingrecorded after the orbital vehicle is over the predefined location onthe Earth. The control system may be configured also to performoperations that include controlling the shutter to record a fifth imageon the image sensor during flight while the predefined location on theEarth is visible to the image sensor, with the fifth image being basedon light from a ground-based calibration system; and calibrating thelight source during flight based on the first image, the second image,the third image, the fourth image, and the fifth image. The examplesystem may include one or more of the following features, either aloneor in combination.

The control system may be configured also to perform operations thatinclude generating a radiance map for a state of the light source basedon the first image, the second image, the third image, the fourth image,and the fifth image. Operation of the light source may be controlled tocalibrate the image sensor. The orbital vehicle may include a shutter toopen to the environment, and the light source and the image sensor arebehind the shutter.

Recording the dark images, the third and fourth images, and the fifthimage may include the following operations, which may or may not beperformed in the following order: controlling the shutter to close torestrict environmental light from reaching the image sensor; recordingthe first image on the image sensor when the shutter is closed and thelight source is off; turning the light source on to illuminate the imagesensor when the shutter is closed; recording the third on the imagesensors when the shutter is closed and the light source is illuminated;turning the light source off; controlling the shutter to open to allowlight from a ground-based calibration system to illuminate the imagesensor; recording the fifth image on the image sensor based on the lightfrom the ground-based calibration system; controlling the shutter toclose to restrict environmental light from reaching the image sensor;turning the light source on to illuminate the image sensor when theshutter is closed; recording the fourth image on the image sensor whenthe shutter is closed and the light source is illuminated turning thelight source off; and recording the second image on the image sensorswhen the shutter is closed and the light source is off.

The light source may be one or more among multiple light sourcesincluded on the orbital vehicle, where different combinations of themultiple light sources define different states. Operations performed bythe control system may include performing the calibration for differentstates that include the light source based on the first image, thesecond image, the third image, the fourth image, and the fifth image.

Calibrating the light source may include the following operations:obtaining a dark bias of the light source based on the first and secondimages; removing the dark bias from the third, fourth, and fifth images;performing a calibration analysis of the sensor based on the fifth imagehaving the dark bias removed to obtain gain coefficients for pixels inthe sensor, where each gain coefficient is for converting a digitalnumber for a pixel into a radiance value; determining average digitalnumbers for pixels in the sensor based on the third and fourth imageshaving the dark bias removed; and applying the gain coefficients to theaverage digital numbers. Applying the gain coefficient may includemultiplying the gain coefficients by the average digital numbers.

The dark bias may include an average dark bias for multiple images. Thecontrol system may be configured to repeat the following operations fortwo or more different spectral bands of the light: recording the darkimages on the image sensor on-board the orbital vehicle during flight,with the dark images including the first and second images; recordingthe third and fourth images during flight; recording the fifth image onthe image sensor during flight while the orbital vehicle is over thepredefined location on the Earth; and generating the radiance map.

The ground-based calibration system may include a plurality of sphericalmirrors disposed upon a uniform background as at least one array ofreflective points, with at least two points of the array reflecting anintensity of directly incident sunlight.

Any two or more of the features described in this specification,including in this summary section, may be combined to formimplementations not specifically described in this specification.

At least part of the systems and processes described in thisspecification may be configured or controlled by executing, on one ormore processing devices, instructions that are stored on one or morenon-transitory machine-readable storage media. Examples ofnon-transitory machine-readable storage media include read-only memory,an optical disk drive, memory disk drive, and random access memory. Atleast part of the systems and processes described in this specificationmay be configured or controlled using a computing system comprised ofone or more processing devices and memory storing instructions that areexecutable by the one or more processing devices to perform variouscontrol operations including control over satellite operations.

The details of one or more implementations are set forth in theaccompanying drawings and the following description. Other features andadvantages will be apparent from the description and drawings, and fromthe claims.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example satellite and its control system,including an exploded view of sensor system components contained in thesatellite.

FIG. 2 is a block diagram of an example specular array for radiometriccalibration (SPARC) system containing convex mirrors having differingradii of curvature, differing geometric patterns of mirror arrays, and asatellite positioned relative thereto.

FIG. 3 is a flowchart showing an example process for calibrating lightsources contained on the satellite of FIG. 1.

FIG. 4 is a top view of different types of images captured duringsatellite orbit, which are used in the example calibration process ofFIG. 3.

FIG. 5 is a flowchart showing example operational image calibrationbased on calibration coefficients generated using the operationsincluded in the calibration process of FIG. 3.

FIG. 6 is a top view showing satellite imaging over, and relative to,Hawaii, which is an example SPARC target location

Like reference numerals in different figures indicate like elements.

DETAILED DESCRIPTION

Described herein are example systems and processes for calibrating,during flight, light sources that are on-board an orbital vehicle. Asatellite is used as an example orbital vehicle in the followingdescription; however, the systems and processes are not limited to usewith satellites. The light sources, which may be lamps, light-emittingdiodes (LEDs), or any other appropriate source of illumination, areconfigured to calibrate image sensors (“sensors”) on a satellite. Thesensors are configured to capture images of the Earth from lightreflected from the Earth.

In this regard, during imaging, the satellite orbits the Earth. Eachorbit is referred to as a path and may run in the north-south direction.Images from the orbit are captured by a detector focal plane arrayincluded on the sensors. Examples of sensor arrays that have been andmay be used on satellites utilizing on-board calibration light sourceassemblies include, but are not limited to, the NASA EO-1 Advanced LandImager (ALI), the MultiSpectral Scanner (MSS) system, the ThematicMapper (TM) and the Operational Land Imager (OLI). Each successive orbitof the satellite may capture images from a different part of the Earth,resulting in individual segmented images of the Earth being recorded byrow (east-west) and path (north-south) coordinates. There are twoprimary types of satellite image sensors, which are defined in terms ofhow they scan: along-track (or “push-broom”) sensors and across-track(or “whisk-broom”) sensors. Example sensor designs have a spatialresolution of 30 meters (m).

Raw images captured by the sensors may be characterized as pixel digitalnumbers (DNs) that are proportional to the scene irradiance illuminatingthe focal plane through an optical system. The example image calibrationprocess described herein converts the DNs to radiance units based ongain coefficients characterized by the on-board calibration system thatcan include light sources. The light source illumination may be producedby heated filaments or LEDs. LEDs may consume less power and may providea better representation of the shape of the solar spectrum.

The sensors may be calibrated during flight using a ground-basedcalibration system. An example of a ground-based calibration system thatmay be used to calibrate the on-board satellite sensors during flight isthe SPARC system described in U.S. Pat. No. 8,158,929 (Schiller), whichis incorporated herein by reference. As noted, the SPARC system enablesabsolute calibration of the sensors, in which an output DN from a sensorpixel is converted into a representation of a physical phenomenon, suchas radiance in this example. The expected radiance values output by theSPARC system may be compared to the on-board calibrator-derived radiancevalues detected by the sensors and the sensor output values may beadjusted to account for any difference between the two.

In some situations, ground-based vicarious solar calibration may not beavailable to the satellite. For example, ground-based SPARC targets orother vicarious targets such as pseudo invariant sites (PICS) may be outof view due to the position of the satellite, clouds, or the time ofday—for example, it may be night. The on-board light sources thereforemay be used to calibrate the sensors. In this regard, as noted, theon-board light sources are calibrated prior to launch of the satellite;however, the light sources are also subject to change in operationfollowing launch and during flight. Accordingly, the light sources mayalso be calibrated during flight to account for any changes in operationthat occurred following launch. The light sources may be recalibratedusing the ground-based calibration system when the ground-basedcalibration system is visible. And later, such as when the ground-basedcalibration system is not available, the light sources may be used tocalibrate the sensors at a higher temporal frequency than possible byvicarious methods. That is, the light sources become a references ofknown radiance illumination to determine if the sensors are changing asthe satellite orbits the Earth, including in areas where ground-basedcalibration is not available. The calibration performed using the lightsources may be absolute, thus making absolute calibration available anytime during flight. Furthermore, the light sources may be smaller thanother types of on-board calibration systems, such as solar diffusers.And, unlike solar diffusers, the light sources do not rely on sunlightor require repositioning of the satellite toward the sun, which canreduce imaging time.

An example satellite system, which is described in more detail below,performs operations during flight—for example, while orbiting theEarth—that include the following. Dark images are recorded on an imagesensor on-board the satellite during flight. The dark images include afirst image recorded before the satellite is over a predefined locationon the Earth, such as a location on the Earth where SPARC targets arevisible to the satellite's image sensor (“SPARC location”), and a secondimage recorded after the satellite is over the SPARC location. Lightsource images are recorded on the image sensor on-board the satelliteduring flight. The light source images include a third image and afourth image that are recorded on the image sensor during flight basedon illumination from the light source that is on-board the satellite.The third image is recorded before the satellite is over the SPARClocation and the fourth image recorded after the satellite was over theSPARC location. A fifth image, nested temporally between the pair ofdark and light source images, is recorded on the image sensor duringflight while the SPARC targets are visible. The fifth image includes thelight from the SPARC targets. Gain coefficients are generated based onthe first through fifth images to calibrate the light source duringflight, as described below.

An example system 10 that may be included on a satellite 12 is shown inFIG. 1. All components of the satellite on which system 10 resides arenot shown. System 10 includes a telescope subsystem 14 for directinglight toward a sensor array 15 that includes multiple spectral imagesensors. In this example, telescope subsystem 14 includes a first (orprimary) mirror 17 to receive light 18 from an object such as thesurface of the Earth and to reflect light 18 to produce first reflectedlight 19. A second (or secondary) mirror 20 receives and reflects thefirst reflected light 18 to produce second reflected light 22. A third(or tertiary) mirror 24 receives and reflects the second reflected light22 to produce third reflected light 25. A fourth (or quaternary) mirror27 receives and reflects the third reflected light 25 towards a focalplane assembly containing the sensor array 15. In some implementations,first mirror 17 includes a reflective surface that is at least partly orwholly a zero or low optical power surface; second mirror 20 includes areflective surface that is at least partly or wholly a zero or lowoptical power surface; third mirror 24 includes a reflective surfacethat is at least partly or wholly a positive optical power surface; andfourth mirror 27 includes a reflective surface that is at least partlyor wholly a positive optical power surface. Optical power is the degreeto which a mirror converges or diverges reflected light. Mirrors thatdiverge light have negative optical power and mirrors that convergelight have positive optical power. Baffles 30, 31, 32, and 33 mayprotect the mirrors from stray light reflections. Telescope subsystem 14is not limited to the optical configuration show in FIG. 1 and mayemploy additional or different optical components.

System 10 includes a shutter 36 that is controllable to open to allowlight 38 from the environment to enter telescope subsystem 14 or toclose to prevent light 38 from the environment from entering telescopesubsystem 14. In some implementations, shutter 36 may be an opaquemechanical structure that is movable into or out of the path of light 38from the environment. Light from the environment may include any lightfrom outside the satellite, such as light from the Sun, the stars, theEarth, or other satellites.

As shown in FIG. 1, light sources 40 are located behind shutter 36 onsatellite 12. As noted, the light sources may be lamps, light-emittingdiodes (LEDs), or any other controllable source of illumination. Thelight sources are configured, arranged, and controllable to shine lightthrough aperture 42 of telescope subsystem 14, which directs the lightto sensor array 15 as shown conceptually in FIG. 1. In someimplementations, light sources 40 include twelve (12)computer-controllable lamps, although only two are shown. The lamps arecontrollable in multiple states. A state includes different combinationsof lamps. For example, for lamps numbered one to twelve, a first statemay include illuminating lamps one, two, and three while the remaininglamps are off; a second state may include illuminating lamps four, five,six, and ten while the remaining lamps are off; and a third state mayinclude illuminating lamps one, four, and twelve, while the remaininglamps are off; and so forth. In some implementations, the lamps arecontrollable in groups of three, where three “on” lamps defines a state;however, that is not a requirement. The lamps are configured toilluminate the sensor array over multiple non-thermal spectral bands,e.g., wavelengths of light. In some implementations, different lamps maybe configured and controllable to illuminate over a limited range ofspectral bands. In some implementations, different lamps may beconfigured and controllable to illuminate over all visible spectralbands.

Operation of satellite 12 is controlled using a control system 44. Amongother things, control system 44 controls operation of shutter 36—forexample to open or to close the shutter, and operation of the lightsources—for example, to turn selected light sources on or off and/or tocontrol their illumination levels in some examples. Control system isalso configured to perform on-board calibration of the sensor array andof light sources using the example techniques described herein.

Control system 44 may include circuitry and/or an on-board computingsystem 45 to control operations of the satellite. The circuitry oron-board computing system is “on-board” in the sense that it is locatedon the satellite itself. On-board computing system 45 may include, forexample, one or more microcontrollers, one or more microprocessors,programmable logic such as a field-programmable gate array (FPGA), oneor application-specific integrated circuits (ASICs), solid statecircuitry, or any appropriate combination of two or more of these typesof processing devices.

In some implementations, on-board components of control system 44 maycommunicate with a remote computing system 46, which may be part of thecontrol system. This computing system is remote in the sense that it isnot located on the satellite itself. For example, control system 44 canalso include computing resources distributed to a remote location—forexample, part of ground operations at one or more ground locations—atleast a portion of which is not on-board the satellite. Commands provideby the remote computing system may be transferred for execution by theon-board computing system. In some implementations, control system 44includes only on-board components. In some implementations, controlsystem 44 includes a combination of on-board components and the remotecomputing system. In some implementations, control system 44 may beconfigured—for example programmed—to implement control functions basedat least in part on input from a person.

The light 38 from the environment that reaches sensor array 15 mayinclude light reflected from the Earth, including light reflected fromSPARC targets arranged on the Earth. In this regard, the SPARC system isdescribed, in part, as follows in U.S. Pat. No. 8,158,929 (Schiller).FIG. 2 conceptually illustrates a perspective view of an example SPARCsystem (or simply “SPARC). SPARC 50 includes spherical mirrors, of whichmirror 52 is an example, disposed upon a uniform low reflectancebackground 54 so as to provide an array of reflective points on theground. In example implementations, the background is an asphaltpavement or a substantially uniform grassy area. At least two points,e.g., mirrors 56 and 58, reflect different intensities of directlyincident sunlight 59 due to their different radii of curvature.Intensities may also be selected by combining different numbers ofmirrors into a single target.

Each mirror, such as mirror 52, has a radius of curvature 62 and adiameter 64. The radius of curvature 62 and the diameter 64 provide afield of regard 66. Collectively, all mirrors of SPARC 50 provide acollective minimum calibratability field of regard. The field of regardrepresents the solid angle of reflected light from the mirror in whichthe image of the sun is visible. When a sensor 68, which is part of asensor array, is to be calibrated, for example when satellite 72 iswithin the minimum calibratability field of regard, calibration canoccur using all features of the SPARC array. The mirrors of SPARC 50 mayhave different radii of curvature and diameter dimensions, and as suchdifferent individual fields of regard. In implementations in which allmirrors are collectively utilized for calibration, the collectiveminimum calibratability field of regard may be determined by thesmallest field of regard produced by a member of the SPARC 50 array

Each mirror 52 may be concave or convex, however, in someimplementations, the mirrors 52 are all convex. As domed structuresrising from the plane of the ground, the convex shape may haveadvantages over concave spherical mirrors such as, but not limited to, areduced likelihood to collect rain, snow, leaves, or other debris thatmight adversely affect the reflective properties of the mirror. Theconvex mirror also produces a virtual image of the Sun, thereby avoidingthe dangers of light concentration by concave mirrors.

In some implementations, the spherical mirrors are subgrouped. In anexample, the SPARC system may include a first subgroup 74 and a secondsubgroup 76. In this example, the mirrors of first subgroup 74 are aboutidentical as shown. Furthermore, as shown in second subgroup 76, in thisexample, at least two mirrors, such as mirrors 56 and 58, have differentradii of curvature. Also, in second subgroup 76, in at least thisexample, at least two mirrors, such as mirrors 56 and 58, have differentdiameters. Generally, the mirrors of SPARC 50 each provide a pointsource target, as recorded by the satellite sensor array, which iscollectively an array of reflective points. In some implementations, themirrors of first subgroup 74 provide for calibration of spatial phasingand the mirrors of second subgroup 76 provide point sources of varyingintensity to fully calibrate the dynamic ranges of the sensors.

In some implementations, SPARC 50 is structured and arranged to orientthe field of regard 67 and therefore the collective minimumcalibratability field of regard towards a sensor 68 be calibrated. Thisis shown in FIG. 1 by example subgroup 78. Such orientation may beachieved by raising one side of each mirror, such as by an adjustable orstatic foot in a single mirror or a panel of mirrors that is fitted withan actuator 79 structured and arranged to actuate the mirrors todynamically orient the collective minimum calibratability field ofregard towards a sensor 68 to be calibrated.

SPARC 50 may include at least one information gatherer 80 structured andarranged to gather atmospheric information. In some implementations,information gatherer 89 is a solar radiometer operable to determinetransmittance (T1) of the atmosphere between the mirrors of SPARC 50 andthe sun 81—indicated by dotted line 89, and used to calculate thetransmittance (T2) of the atmosphere between the mirrors of SPARC 100and sensor to be calibrated 58—indicated by dotted line 82.

In some implementations, information gatherer 80 also includes a sensorlocator structured and arranged to determine the location of the sensorto be calibrated 68. The location information regarding the sensor maybe provided in the metadata that comes from the sensor 68 system itself.Therefore, in an example, the element of the information gatherer 80that determines the location of the sensor 68 is not so much trulydetermining the location as it is simply reading and interpreting theinformation provided directly by sensor 68. In an example, a separatesensor locator, such as radar or triangulation, may be employed. Suchlocation and transmittance information may be used in determining theintensity of the reflected directly incident sunlight as provided tosensor 68 to be calibrated.

In SPARC 50, mirrors 52 advantageously provide “solar stars” at theground site, which are seen by image sensor 68 as a virtual image of thesun produced by each mirror. The solar irradiance measured inwatts-per-meter squared (watts/m²) at the time of incidence is convertedto an intensity source measured in watts-per-steradian (watt/sr) thatilluminates any surface or sensor input aperture with an irradiance thatfollows the inverse square law. The intensities for each mirror 52 maybe constant over all illumination and view angles making them versatilefor multiple sensors, multiple view angles, and multiple atmosphericconditions. If a mirror has a radius of curvature that is precise, auniform intensity is produced within a solid angle cone with an angularwidth determined by the diameter of the mirror's reflecting surface andthe radius of curvature. As long as an overflying sensor can see thereflection of the sun, an effective radiance response can be determinedand used for on-board sensor calibration.

In some implementations, calibration need not be performed with respectto a target on the Earth such as a SPARC target or PICS site. Forexample, in some implementations, the calibration target may be acelestial object imaged just above the Earth's limb or inserted into adeep space image. The celestial object may be any appropriate sourcesuch as the Moon or stars having a known absolute radiance or intensityat the time the target is imaged and the images described herein arecollected.

Referring to FIGS. 1 and 3, a process 100 is shown for calibrating lightsources 40. Process 100 may be repeated for all or some states of thelight sources. In addition, process 100 may be repeated for all or somewavelengths of light received at sensor array 15. Process 100 may beperformed at least in part by, or controlled at least in part by, thecontrol system 44 described herein.

In some implementations, process 100 is performed using SPARC targetslocated on Mauna Loa, Hi. 99 (FIG. 4). There, the SPARC targets arelocated at an elevation of over 10,000 feet (3048 meters), which meansthat they are above most cloud cover. In addition, in some exampleswhere a satellite system only processes land imaging, satellite 12operationally does not deliver imaging recorded over the ocean. In anexample, there is 13 minutes of dead time over the ocean prior toreaching the SPARC targets and five minutes of dead time over the oceanafter passing the SPARC targets. During this dead time, satellite 12 isconfigured not to perform imaging. Accordingly, these times whensatellite 12 is over the ocean may be used to calibrate the lightsources without disrupting imaging. In process 100, this time isreferred to as “over-ocean” time, whereas times when the satellite is inposition to image the SPARC targets are referred to as “over-target”time.

While satellite 12 is over-ocean, shutter 36 is closed (101) to restrictenvironmental light 38 from reaching sensor array 15. For example,closing shutter 36 may prevent light from the Earth, the Sun, and/or thestars from reaching sensor array 15. At this time, light sources 40 areturned-off (in which case, the light sources do not illuminate or godark), also preventing light from the light sources from reaching thesensor array. While the satellite is over-ocean, shutter 36 is closed,and light sources 40 are off, a first dark image is recorded on sensorarray 15. The image is referred to as a “dark image” because it iscaptured (102) while there is no illumination of, or minimalillumination of, image sensors in sensor array 15. Referring to FIG. 4,the first dark image (1) 92 may be stored in computer memory 49 on thesatellite, for example. The dark image characterizes the readout biasfor all the sensors.

While satellite 12 is over-ocean and shutter 36 is closed, one ormore—for example, three—of the light sources are turned on (103). Thecombination of turned-on light sources is referred to as a state of thelight sources, as explained above. Because the shutter remains closed,light from light sources 40 illuminates sensor array 15. In someexamples, only light from light sources 40 illuminates sensor array 15at this time. A first light source image is recorded (104) on sensorarray 15 when shutter 36 is closed and light sources 40 are illuminated.This image is referred to as a “light source image” because it is animage solely or primarily of light from light sources 40. Referring toFIG. 4, the first light source image (2) 93 may be stored in computermemory 49 on the satellite, for example.

As satellite 12 proceeds along its orbital path, satellite 12 moves fromover the ocean to over land—in this example, over land in view of theSPARC targets on Mauna Loa, Hi. 99. Prior to the satellite beingover-target, light sources 40 are turned-off and shutter 36 is opened(105). In this configuration, light from the environment can entertelescope subsystem 14. Accordingly, in this configuration, whilesatellite 12 is over-target, light from the ground-based calibrationsystem—in this example, the SPARC targets—is allowed to illuminatesensor array 15. Images—referred to as SPARC images—are recorded (106)on sensor array 15 based on light reflected form the SPARC targets.Referring to FIG. 4, one or more SPARC images (3) 94 may be stored incomputer memory 49 on the satellite, for example.

As satellite 12 proceeds along its orbital path, the satellite movesfrom over land to over the ocean. Prior to or while the satellite isover-ocean, shutter 36 is closed (108) as above to restrictenvironmental light from reaching sensor array 15. As explainedpreviously, closing shutter 36 may prevent environmental light from theEarth, the Sun, and the stars from reaching sensor array 15. At thistime, light sources 40 are turned-on (114) to illuminate sensor array 15when shutter 36 is closed (108). This results in the same configurationas described above. At this time, a second light source image isrecorded (109) on sensor array 15. The second light source image, likethe first light source image described previously, is an image solely orprimarily based on light from a state of light sources 40. Referring toFIG. 4, the second light source image (4) 95 may be stored in computermemory 49 on the satellite, for example.

While the satellite is over-ocean and while shutter 36 is closed, lightsources 40 (for example, all light sources) are turned off (110). Whilethe satellite is over-ocean, shutter 36 is closed, and light sources 40are off, a second dark image is recorded (111) on the image sensor. Asnoted, this dark image is captured while there is no illumination of, orminimal illumination of, image sensors in sensor array 15. Referring toFIG. 4, the second dark image (5) 96 may be stored in computer memory 49on the satellite, for example.

Process 100 includes calibrating (112) the light sources based on thefirst and second dark images captured before and after land, the firstand second light source images captured before and after land, and theimages captured from the SPARC targets. Calibration may be performed forall, some, or one state of the light sources. A summary of an examplecalibration process that may be performed in operation 112 includes thefollowing. A pair of light source DN values and dark image DN values areaveraged for each spatial pixel to obtain by interpolation the DN lightsource and dark image response relative to the time of SPARC imagecollection. Subtracting the dark image from the light source image andperforming a flat field correction creates the dark subtracted DN lightsource response image to be calibrated. An analysis of a SPARC imageprovides the DN per-unit-radiance gain coefficient that applies to allpixels in the dark subtracted light source image. Multiplying each pixelDN value in the dark subtracted light source image by the SPARC gaincoefficient converts the light source DN image to a radiance imagecompleting the calibration and generating a radiance map (113) producedby the light source at the focal plane. The radiance map includesradiance values for a given wavelength of light for each pixel in thesensor array illuminated by a state of the light sources. For example,there may be a one-to-one mapping of pixels to radiance values in theradiance map. It is assumed that the relative pixel non-uniformitycorrection (NUC) coefficients used to perform a flat field correctionwere determined independently prior to the start of process 100.

Calibration (112) may be performed for each light sources state andwavelength as noted, although one state and wavelength is described inthe following. Example calibration operations (112) are shown in FIG. 5.To perform a calibration, the dark bias of light sources 40 is obtained(120) based on the first and second dark images. The dark bias includesthe amount of light that registers on sensor array 15 when no light orminimal light is applied to sensor array 15. Each image, including thedark images, from the sensors is defined by DNs, with each DN beingproduced by a single pixel on an image sensor. In this example, the darkbias may be obtained by averaging the DNs of the first and second darkimages. In this example, the resulting dark bias is then removed (121)from the first and second light source images and from the imagescaptured from the SPARC targets. Removing the dark bias may includesubtracting the averaged DNs from the DNs for each of the other images.That is, subtracting may include subtracting the averaged DNs from thedark images from the DNs for each of the light source images and the DNsfor the SPARC target images. This may be done for each light sourcestate and wavelength. The flat field correction described previously maybe performed on the resulting images.

A calibration analysis of sensor array 15 is performed (122) using theSPARC images having the dark bias removed to obtain gain coefficientsfor pixels in the sensors. The calibration analysis may be performed toidentify a difference between an expected sensor output using knownvalues from the SPARC targets and the actual sensor output. The gaincoefficients account for this difference. For example, the gaincoefficients may be applied to the DNs of each pixel to account foraberrations in the sensor operation. There may be one gain coefficientper pixel, which is used to convert a DN for that pixel into a radiancevalue that matches an expected radiance value.

Average DNs for pixels in each sensor of the sensor array are determined(123) based on the first and second light source images having the darkbias removed. For example, DNs for the light source images with the darkbias removed may be averaged so that the resulting average valuecoincides with the location at which images of the SPARC targets werecaptured. For example, referring to FIGS. 4 and 6, assume that the firstlight source image 92 is captured at location 130, the image(s) 94 ofthe SPARC targets are captured at location 131, and the second lightsource image 95 is captured at location 132. Locations 130 and 132 maybe equidistant from location 131 at which image(s) 94 of the SPARCtargets are captured. In this example, the DN for the first light sourceimage is 2020 and the DN for the second light source image is 2000. Thetwo DNs are averaged to produce an averaged DN of 2010. Because thelocation where the image of the SPARC targets is captured is mid-waybetween the locations where the light source images are captured, theaverage DN value of 2010 is assumed for the light sources at thelocation where the image of the SPARC targets is captured. To calibratethe on-board light source, the gain coefficient for each pixeldetermined using the SPARC targets above is applied (124) to theaveraged DN for that pixel obtained using the first and second lightsource images with dark bias removed, as described above. Applying thegain coefficient may include multiplying the gain coefficients by theaveraged DNs of individual pixels of the light source images. Absoluteradiance maps for each state of the light sources may be generated usingthe products of the averaged DN values and the gain coefficients.

Although the systems and processes herein have been described in thecontext of a satellite, they may be used with any appropriate aerial ororbital vehicle that is configured to obtain images of the Earth orother celestial body that includes SPARC targets or other ground-basedcalibration systems.

All or part of the systems and processes described in this specificationand their various modifications may be configured or controlled at leastin part by one or more computers using one or more computer programstangibly embodied in one or more information carriers, such as in one ormore non-transitory machine-readable storage media. A computer programcan be written in any form of programming language, including compiledor interpreted languages, and it can be deployed in any form, includingas a stand-alone program or as a module, part, subroutine, or other unitsuitable for use in a computing environment. A computer program can bedeployed to be executed on one computer or on multiple computers at onesite or distributed across multiple sites and interconnected by anetwork.

Actions associated with configuring or controlling the systems andprocesses can be performed by one or more programmable processorsexecuting one or more computer programs to control all or some of thewell formation operations described previously. All or part of thesystems and processes can be configured or controlled by special purposelogic circuitry, such as, an FPGA (field programmable gate array) and/oran ASIC (application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only storagearea or a random access storage area or both. Elements of a computerinclude one or more processors for executing instructions and one ormore storage area devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom, or transfer data to, or both, one or more machine-readable storagemedia, such as mass storage devices for storing data, such as magnetic,magneto-optical disks, or optical disks. Non-transitory machine-readablestorage media suitable for embodying computer program instructions anddata include all forms of non-volatile storage area, including by way ofexample, semiconductor storage area devices, such as EPROM (erasableprogrammable read-only memory), EEPROM (electrically erasableprogrammable read-only memory), and flash storage area devices; magneticdisks, such as internal hard disks or removable disks; magneto-opticaldisks; and CD-ROM (compact disc read-only memory) and DVD-ROM (digitalversatile disc read-only memory).

Elements of different implementations described may be combined to formother implementations not specifically set forth previously. Elementsmay be left out of the systems described previously without adverselyaffecting their operation or the operation of the system in general.Furthermore, various separate elements may be combined into one or moreindividual elements to perform the functions described in thisspecification.

Advantages of the example implementations described herein may includethe ability to collect all needed image data for in-flightre-calibration of one or more on-board light source such as one or morelamps, as described herein, while almost simultaneously eliminatingpotential biases due to temporal drift in the sensor system'sradiometric response between image collections that occur in standardEarth remote sensing concepts of operations (ConOps).

Other implementations not specifically described in this specificationare also within the scope of the following claims.

What is claimed is:
 1. A method of calibrating a light source that ison-board an orbital vehicle during flight, the method comprising:recording dark images on an image sensor on-board the orbital vehicleduring flight, the dark images including a first image recorded beforethe orbital vehicle is over a predefined location on the Earth and asecond image recorded after the orbital vehicle is over the predefinedlocation on the Earth; recording a third image and a fourth image on theimage sensor during flight based on illumination from the light sourcethat is on-board the orbital vehicle, the third image being recordedbefore the orbital vehicle is over the predefined location on the Earthand the fourth image being recorded after the orbital vehicle is overthe predefined location on the Earth; recording a fifth image on theimage sensor during flight while the predefined location on the Earth isvisible to the image sensor, the fifth image being based on light from aground-based calibration system; and calibrating the light source duringflight based on the first image, the second image, the third image, thefourth image, and the fifth image.
 2. The method of claim 1, furthercomprising: generating a radiance map for a state of the light sourcebased on the first image, the second image, the third image, the fourthimage, and the fifth image.
 3. The method of claim 1, furthercomprising: controlling operation of the light source to calibrate theimage sensor.
 4. The method of claim 1, wherein the orbital vehiclecomprises a shutter to open to the environment and wherein the lightsource and the image sensor are behind the shutter; and whereinrecording the dark images, the third and fourth images, and the fifthimage comprises: closing the shutter to restrict environmental lightfrom reaching the image sensor; recording the first image on the imagesensor when the shutter is closed and the light source is off; turningthe light source on to illuminate the image sensor when the shutter isclosed; recording the third on the image sensors when the shutter isclosed and the light source is illuminated; turning the light sourceoff; opening the shutter to allow light from a ground-based calibrationsystem to illuminate the image sensor; recording the fifth image on theimage sensor based on the light from the ground-based calibrationsystem; closing the shutter to restrict environmental light fromreaching the image sensor; turning the light source on to illuminate theimage sensor when the shutter is closed; recording the fourth image onthe image sensor when the shutter is closed and the light source isilluminated; turning the light source off; and recording the secondimage on the image sensors when the shutter is closed and the lightsource is off.
 5. The method of claim 1, wherein the light source isamong multiple light sources included on the orbital vehicle, wheredifferent combinations of the multiple light sources comprise differentstates; and wherein the method comprises performing the calibrating fordifferent states that include the light source based on the first image,the second image, the third image, the fourth image, and the fifthimage.
 6. The method of claim 1, wherein calibrating comprises:obtaining a dark bias of the light source based on the first and secondimages; removing the dark bias from the third, fourth, and fifth images;performing a calibration analysis of the image sensor based on the fifthimage having the dark bias removed to obtain gain coefficients forpixels in the sensor, each gain coefficient for converting a digitalnumber for a pixel into a radiance value; determining average digitalnumbers for pixels in the image sensor based on the third and fourthimages having the dark bias removed; and applying the gain coefficientsto the average digital numbers.
 7. The method of claim 6, whereinapplying comprises multiplying the gain coefficients by the averagedigital numbers.
 8. The method of claim 1, wherein the dark biascomprises an average dark bias for multiple images.
 9. The method ofclaim 1, further comprising, repeating the following operations for twoor more different spectral bands of the light: recording the dark imageson the image sensor on-board the orbital vehicle during flight, the darkimages including the first and second images; recording the third andfourth images during flight; recording the fifth image on the imagesensor during flight while the orbital vehicle is over the predefinedlocation on the Earth; and generating the radiance map.
 10. The methodof claim 1, wherein the ground-based calibration system comprises aplurality of spherical mirrors disposed upon a uniform background as atleast one array of reflective points, at least two points of the arrayreflecting an intensity of directly incident sunlight.
 11. A system onan orbital vehicle, the system comprising: an image sensor for capturingimages based on incident light; a light source to illuminate the imagesensor; a shutter that is controllable to allow environmental light toreach the image sensor to prevent the environmental light from reachingthe image sensor; and a control system to perform operations comprising:controlling the shutter to record dark images on the image sensoron-board the orbital vehicle during flight, the dark images including afirst image recorded before the orbital vehicle is over a predefinedlocation on the Earth and a second image recorded after the orbitalvehicle is over the predefined location on the Earth; controlling theshutter to record a third image and a fourth image on the image sensorduring flight based on illumination from the light source that ison-board the orbital vehicle, the third image being recorded before theorbital vehicle is over the predefined location on the Earth and thefourth image being recorded after the orbital vehicle is over thepredefined location on the Earth; controlling the shutter to record afifth image on the image sensor during flight while the predefinedlocation on the Earth is visible to the image sensor, the fifth imagebeing based on light from a ground-based calibration system; andcalibrating the light source during flight based on the first image, thesecond image, the third image, the fourth image, and the fifth image.12. The system of claim 11, wherein the operations comprise: generatinga radiance map for a state of the light source based on the first image,the second image, the third image, the fourth image, and the fifthimage.
 13. The system of claim 12, wherein operation of the light sourceis controlled to calibrate the image sensor.
 14. The system of claim 11,wherein the orbital vehicle comprises a shutter to open to theenvironment, and where the light source and the image sensor are behindthe shutter; and wherein recording the dark images, the third and fourthimages, and the fifth image comprises: controlling the shutter to closeto restrict environmental light from reaching the image sensor;recording the first image on the image sensor when the shutter is closedand the light source is off; turning the light source on to illuminatethe image sensor when the shutter is closed; recording the third on theimage sensors when the shutter is closed and the light source isilluminated; turning the light source off; controlling the shutter toopen to allow light from a ground-based calibration system to illuminatethe image sensor; recording the fifth image on the image sensor based onthe light from the ground-based calibration system; controlling theshutter to close to restrict environmental light from reaching the imagesensor; turning the light source on to illuminate the image sensor whenthe shutter is closed; recording the fourth image on the image sensorwhen the shutter is closed and the light source is illuminated; turningthe light source off; and recording the second image on the imagesensors when the shutter is closed and the light source is off.
 15. Thesystem of claim 11, wherein the light source is among multiple lightsources included on the orbital vehicle, where different combinations ofthe multiple light sources comprise different states; and wherein theoperations comprise performing the calibrating for different states thatinclude the light source based on the first image, the second image, thethird image, the fourth image, and the fifth image.
 16. The system ofclaim 11, wherein calibrating the light source comprises: obtaining adark bias of the light source based on the first and second images;removing the dark bias from the third, fourth, and fifth images;performing a calibration analysis of the image sensor based on the fifthimage having the dark bias removed to obtain gain coefficients forpixels in the sensor, each gain coefficient for converting a digitalnumber for a pixel into a radiance value; determining average digitalnumbers for pixels in the image sensor based on the third and fourthimages having the dark bias removed; and applying the gain coefficientsto the average digital numbers.
 17. The system of claim 11, whereinapplying comprises multiplying the gain coefficients by the averagedigital numbers.
 18. The system of claim 11, wherein the dark biascomprises an average dark bias for multiple images.
 19. The system ofclaim 11, wherein the control system is configured to repeat thefollowing operations for two or more different spectral bands of thelight: recording the dark images on the image sensor on-board theorbital vehicle during flight, the dark images including the first andsecond images; recording the third and fourth images during flight;recording the fifth image on the image sensor during flight while theorbital vehicle is over the predefined location on the Earth; andgenerating the radiance map.
 20. The system of claim 11, wherein theground-based calibration system comprises a plurality of sphericalmirrors disposed upon a uniform background as at least one array ofreflective points, at least two points of the array reflecting anintensity of directly incident sunlight.